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school -improvement projects. Those with less supportive leaders struggled to 
continue their involvement in Quest or terminated their participation in the 
project. According to Quest staff, the use of the network to support 
continuous school improvement has been a fruitful strategy. One benefit had 
been the way in which practitioners have been able to learn from each other 
rather than from researchers, state department of education staff, or other 
entities external to the life of a school. Quest staff reported that the 
financial and logistical support they offered participants was critical to 
the continued involvement of some schools. Project staff and school teams 
made significant efforts to include parents and students in their Quest 
participation and school -improvement initiatives. These efforts succeeded to 
various degrees, though some parents continued to report feeling peripheral 
to their school's undertakings. Student voice proved to be a valuable 
addition to the network. The participation of community members, however, was 
quite limited, as schools struggled to find meaningful ways in which to 
involve them. Appendixes include a brochure and framework for improvement, 
and a completed evaluation standards checklist. (Contains 47 references.) 
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EXECUTIVE SUMMARY 



Based upon principles of inquiry, collaboration, and action research, the Quest project at 
AEL has supported and investigated ongoing school improvement efforts since 1 996 through twice- 
yearly conferences; summer symposia on topics of interest to participating schools; a Scholars 
program in which participants collaborate with Quest staff in research efforts; visits to participating 
schools; communication via listserv and mailings; and the establishment of a Quest network of 
schools. 

As the funding cycle draws to a close, this report summarizes lessons learned from the Quest 
project. Data sources include a group interview with Quest staff conducted on October 18, 2000, 
formative evaluation reports, summative evaluation case studies, and project research reports. 

One lesson learned concerned the importance of school leaders to the success of reform 
implementation. Schools with supportive leadership found ways to attend various network events 
and enact school improvement projects. Those with less supportive leaders struggled to continue 
their involvement in Quest or terminated their participation in the project. 

According to Quest staff, the use of the network to support continuous school improvement 
has been a fruitful strategy. One benefit has been the way in which practitioners have been able to 
learn from each other rather than from researchers, state department of education staff, or other 
entities external to the life of a school. Characterized by some researchers as professional learning 
community (e.g., Hord, 1997; Ryan, 1995), Quest participants modeled strategies for each other, 
shared resources, and discussed issues of educational significance. They met at regular intervals 
throughout year, at fall and spring rallies and summer symposia. Fonnative evaluation data suggest 
that participants forged supportive and productive professional friendships over the course of their 
participation in Quest. 

However, the use of a regional network increased the cost of participation for some schools. 
Quest staff reported that the financial and logistical support they offered participants was critical to 
the continued involvement of some schools. 

Project staff and school teams made significant efforts to include parents and students in their 
Quest participation and school improvement initiatives. These efforts succeeded to various degrees, 
although some parents continued to report feeling peripheral to their school’s undertakings. Student 
voice proved to be a valuable addition to the network. The participation of community members, 
however, was quite limited, as schools struggled to find meaningful ways in which to involve them. 

An issue for further consideration by Quest staff concerns the nonprescriptive nature of the 
project, which was compelling to some schools but confusing to others. Should they undertake 
similar work in the future, project staff may want to consider how to mediate such confusion while 
maintaining the flexibility to allow schools to find improvement strategies that meet their local 
needs. 



ii 
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INTRODUCTION 



Research Foundation of AEL’s Quest Project 

Based upon principles of inquiry, collaboration, and action research, the Quest project has 
supported and investigated ongoing school improvement efforts since 1996 through twice-yearly 
conferences; summer symposia on topics of interest to participating schools; a Scholars program in 
which participants collaborate with Quest staff in research efforts; project staff visits to participating 
schools; communication via listserv and mailings; and the establishment of a Quest network of 
schools. The Quest framework for continuous school improvement is included in Appendix A. 

The project is grounded in school change literature suggesting that subjectivity and personal 
growth are essential to the change process (Fullan, 1991). In other words, significant change cannot 
occur unless it has some meaning to the individuals responsible for its realization. This distinctly 
constructivist approach to school reform has implications for how external facilitation agencies, such 
as AEL, introduce, describe, and support change strategies to school personnel. For instance, Quest 
staff chose to include time and methods for facilitating individual reflection on practice during 
project events. 

Yet because individual development takes place within a variety of social contexts, including 
school communities, staff designed the Quest network with attention to the ways shared vision, 
goals, and sense of community support ongoing school improvement (Barth, 1990; Hord, 
Rutherford, Huling-Austin, & Hall, 1987; Postman, 1995; Sergiovanni, 1994). Similarly, school 
culture may impede or enhance significantly the viability of school improvement work (Richardson, 
1996; Ryan, 1995). If a school community shares certain norms, such as self-evaluation, curiosity, 
proactivity, and high performance expectations, reform efforts are hypothesized to fare better than 
those in school cultures that do not possess such norms. 

This perspective has had implications for the Quest project. Network staff often grounded 
the presentation of novel change strategies in stories detailing other network schools’ use of such 
approaches. Many techniques supporting continuous improvement introduced by project staff were 
intended to nurture the school context. The Protocol process, for instance, is a structured means for 
faculty to discuss student work collaboratively, at once focusing teacher attention on instruction and 
encouraging the development of professional community, shared understanding, and collective 
purpose. 

Other research suggests that school administrators must assume a collaborative role in 
decision-making if reform efforts are to succeed (van der Bogert, 1998), and that instructional and 
curricular goals must be informed by a diverse contingent of school stakeholders, including parents, 
students, and community members (Barth, 1990; Sergiovanni, 1994). As a result of this research, 
Quest staff invited school administrators, teachers, parents, and high school students to network 
gatherings and presented strategies to enable shared decision making. 

Quest staff were also attuned to literature suggesting that honoring the purpose of education 
enhances school change, in part by connecting school staff to the meaning of their work. Wiggins 
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(1993), for instance, argues that assessment strategies ought to serve multiple ends, not the least of 
which is to provide information for ongoing teaching and learning. In other words, assessment, along 
with any other educational practice, should support the enrichment of students’ intellectual lives; 
assessment for its own sake or for the satisfaction of mandate alone may not contribute to the 
ultimate aim of education. In the final reckoning, education generally and reform endeavors 
specifically need to nurture a host of attributes enabling students to make use of their education to 
lead thoughtful, productive lives (Perkins, 1995; Postman, 1995). 

In sum, the research upon which Quest is based has had ramifications for ‘what’ the project 
has been and ‘how’ the project has been implemented. Opportunities for writing about education and 
change were structured to give network participants the otherwise rare chance in their busy 
professional lives to reflect on their own practice. Storytelling, group activities, and the network 
itself allowed participants occasion to exchange ideas, strategies, and struggles, as well as to forge 
new professional friendships. School teams were encouraged to invite diverse members of their 
school communities to participate in Quest, and discussions during network events centered squarely 
on the meaning of education and techniques to support such meaning. 

Quest Activities 

In the summer of 1996, Quest staff at AEL began working with teams from school 
communities in three West Virginia county school districts to invigorate efforts for continuous 
school improvement, using a variety of techniques for gathering input from all those with a stake in 
their local schools (Howley-Rowe, 1998a). This first learning community, called Leadership to 
Unify School Improvement Efforts (LUSIE), consisted of school teams including students, teachers, 
administrators, parents, and community members. Ultimately, this group wrote individual school 
visions and improvement plans, and co-authored (with AEL) Creating Energy for School 
Improvement ( 1 997), a supplemental guide for those poised to write their own state-mandated school 
improvement plans. 

Quest staff also were committed to creating learning communities devoted to exploring 
continuous school improvement across the AEL region of Kentucky, Tennessee, Virginia, and West 
Virginia. Hence, staff scheduled a pilot Inquiry Into Improvement conference in April 1997 for 
selected high schools in the region. Schools were selected in several ways. Some schools were 
recommended for the Quest experience by central office staff or school administrators. Other schools 
were asked to join Quest because they had participated in previous AEL programs. Still other 
schools were invited because Quest staff believed they were primed for the kind of collaborative 
inquiries into school improvement that Quest was designed to provide. 

In October 1997, in Roanoke, Virginia, another conference was held for designated high 
schools in the AEL region, this time with an explicit emphasis on forming and nurturing a network 
of schools (Howley-Rowe, 1998b). A similar conference was held in Nashville, Tennessee, for 
designated elementary schools in November 1997 (Howley-Rowe, 1998c). In order to facilitate the 
development of a Quest school network and to continue to encourage continuous school 
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improvement efforts within network schools, staff planned a sequence of events in 1 998 following 
these initial conferences. Dissatisfied with the conventional and prescriptive connotation of the word 
conference, Quest staff chose to call these network meetings rallies. Thus, all events previously 
called conferences are now termed rallies. 

The high school network met a second time on February 8-10, 1998, at the Pipestem State 
Park Resort in West Virginia (Howley-Rowe, 1998d), following which the elementary school 
network participated in a rally on February 22-24, 1998, in Lexington, Kentucky (Howley-Rowe, 
1 998e). During the summer, 1 1 network members participated in the Quest Scholars Program, 
meeting at a colloquium in Charleston, West Virginia, on July 16-18, 1998, to collaborate with 
project staff in ongoing efforts to conceptualize, design, and research Quest (Howley-Rowe, 1 998f). 
Finally, in August, network members and other educators in AEL’s region participated in a 
symposium on assessment of student work (Howley-Rowe, 1998g). 

From the high school network rally in October 1 997 to the August 1 998 summer symposium, 
Quest staff hosted six network events. The Quest network contained an essentially stable 
membership, although there were differences in the number of school teams that attended each event 
and in the frequency that school teams attended gatherings. Project staff recently investigated this 
phenomenon, finding that administrative support for participation in the network was the factor 
reported to be most important to schools’ initial and sustained involvement in Quest (Howley-Rowe, 
1999a). 



Beginning their second year of network activity, Quest staff invited the elementary and high 
school networks to attend a rally together on November 2-3, 1 998, at the Glade Springs Resort, near 
Daniels, West Virginia (Howley-Rowe, 1999b). Approximately half of the Quest Scholars met on 
November 1, 1998, to plan with project staff several rally activities. Scholars from the high school 
network met for three hours on February 14, 1999, prior to a high school network rally held on 
February 15-16 in Roanoke, Virginia (Howley-Rowe, 1999c). A similar rally was held for 
elementary network members on February 22-23, 1999, in Lexington, Kentucky (Howley-Rowe, 
1999d). 



A second Scholars colloquium was convened from July 12-15, 1999, at Mountain Lake 
Resort, Virginia (Howley-Rowe, 1 999d). The primary purpose of this colloquium was for Quest staff 
and Scholars to collaborate in evaluating and writing about the project, ultimately contributing 
written pieces to a book about the Quest network. In addition, a second summer symposium was 
convened in Gatlinburg, Tennessee, July 26-27, 1999 (Parrish & Howley-Rowe, 2000). 

The third year of Quest events began with two rallies and a Scholars meeting in November 
1999 in Bristol, Virginia. A rally for elementary schools was conducted from November 11-12, 
1999. On November 13-14, Scholars met to discuss writing about their experiences with school 
improvement during Quest and to consider the development or revision of several Quest instruments. 
And ahigh school rally was held November 15-16. Network high schools met again from February 
14-15, 2000 in Roanoke, Virginia. Elementary schools participated in a rally from February 17-18, 
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2000 in Lexington, Kentucky. Evaluation of these events was not conducted as staff turned their 
efforts to summative evaluation of the project; Quest and the 1996-2000 REL contract funding the 
project would come to an end in November 2000. Summative evaluation was conducted in the form 
of case studies of four schools, one per each AEL state (Howley-Rowe, 2000a-d). Formative and 
summative evaluation design and methods are described in the next section of this report. 
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METHOD 

A significant amount of formative evaluation of the Quest project has been conducted, 
resulting in 1 2 evaluation reports. Three research reports, a descriptive case study of a Quest process, 
and two technical reports were also completed during the life of the project. One report (Meehan, 
Orletsky & Sattes, 1997) described the field test of an instrument developed to measure the concept 
of professional learning community in schools. The study revealed that, although the instrument did 
not assess distinct subscales, it did measure with validity and reliability the overall construct of 
professional learning community. Another report (Cowley, 1999) summarized results of a study of 
the relationship between professional learning community in Quest schools and teacher efficacy, 
concluding that there was little correlation between the two. The third report (Howley-Rowe, 1 999a) 
described the results of a study of the factors Quest schools reported to have been most important 
to their initial and sustained involvement in the project. Findings included significant differences 
between school levels and between degrees of engagement with the project. Qualitative results 
indicated that building-level leadership was the factor respondents thought was most important to 
their schools’ initial and ongoing participation. The descriptive case study (Howley-Rowe, 1999e) 
summarized the implementation of the Data in a Day (DIAD) process at Xavier Senior High School, 
including follow-up of the outcomes of the strategy. Four summative case studies were conducted, 
as well, describing in-depth analyses ofproject outcomes in four disparate network schools (Howley- 
Rowe, 2000a-d). In addition, two technical reports (Wiersma, forthcoming, 2000) describe the 
validation of an instrument, based on the Quest framework for continuous improvement, designed 
to measure educators’ perceptions of factors that influence success with school improvement. 

Additionally, Quest staff engaged in reflection and discussion about the project as it evolved, 
using a form of evaluation called participative assessment (Walsh & Sattes, 1995). In this view of 
evaluation, intentional and integrated assessment is part of an effective learning cycle supporting 
individual development. Moreover, as the individual appraises his or her own learning, the larger 
group to which the individual belongs is enhanced by each individual’s growth and insight. If 
assessment is conducted regularly and with commitment, the group may ultimately develop as a 
learning community. Hence, Quest staff debriefed following project events, exchanged e-mail and 
phone calls, participated in staff retreats to discuss significant issues and decisions, and answered 
e-mail prompts from the evaluator. Making meaning of the project’s development was central to 
these interactions and reflections. 

Participants from network schools were also encouraged and invited to engage in 
participative assessment. Storytelling was one strategy supporting such self-appraisal; throughout 
the project, various team members were asked to share during Quest gatherings their understanding 
of their schools’ trajectory of continuous improvement. Journal writing and structured dialogue 
techniques were also used as means of assessment. School staff were encouraged to undertake action 
research and contribute written accounts of their schools’ development to various Quest products. 
Although accessed little by network members, a listserv was provided and moderated by proj ect staff 
to facilitate both individual reflection and network communication. 
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The purpose of this narrative is to articulate some of the lessons learned by Quest staff from 
their experiences of developing, facilitating, and nurturing the project between 1996 and 2000. To 
this end, all evaluation and research reports, and staff reflections and communications, were analyzed 
by theme. In addition, the evaluator will consider and report some of the challenges and rewards of 
assessing a project such as Quest. 

A wide variety of data was collected during the course of the project. Formative evaluation 
included both qualitative and quantitative approaches. An advantage of such triangulation of data 
is that it provides a more comprehensive description of the objects of study than might be rendered 
by use of a single research method. Using several data sources in order to corroborate theses is what 
Brewer and Hunter (1989) call “multimethod research.” This approach posits that the strengths of 
each method will compensate for the weaknesses in others, ultimately providing a more complete 
account of that being studied. In general, formative evaluation of network events included 
questionnaires with closed response option quantitative items and open-ended qualitative questions. 
Semi-structured interviews were conducted, and the evaluator engaged in participant observation 
during project meetings as well. 

Participant observation (Becker & Geer, 1957; Emerson, 1983; Glazer, 1972; Hammersley 
& Atkinson, 1983; Miles & Huberman, 1994) is a method highly suited “for studying processes, 
relationships among people and events, the organization of people and events, continuities over time, 
and patterns” (Jorgensen, 1 989, p. 1 2). Furthermore, consistent with the Quest paradigm, participant 
observation involves “a flexible, open-ended, opportunistic process and logic of inquiry through 
which what is studied constantly is subject to redefinition based on field experience and observation” 
(Jorgensen, 1989, p. 23). This method “is a commitment to adopt the perspective of those studied 
by sharing in their . . . experiences” (Denzin, 1989, p. 156), thereby enabling researchers to evaluate 
how an event or process appears and feels to participants. And finally, participant observation places 
the evaluator squarely in the field, rather than in the office or on the phone, allowing for the 
collection of richer, more directly acquired data (Patton, 1980). 

Denzin (1989) describes four variations in participant observation strategies: the complete 
participant, the participant as observer, the observer as participant, and the complete observer (pp. 
162-65). The evaluator played a role more akin to the participant as observer, participating in 
ongoing project activities as appropriate but not concealing data collection. 

Interviews and evaluation questionnaires enabled the evaluator to examine ways in which 
they corroborated or contradicted participant observation findings. Moreover, such strategies allowed 
respondents the opportunity to express their assessments of the project and to appraise the extent to 
which project goals were met. 

Summative evaluation of Quest was similarly multimethod in approach. The evaluator and 
a trained Quest consultant conducted data collection site visits to four case study schools during the 
spring of 2000. Semi-structured individual interviews were conducted with Quest team members, 
as were focus groups with school staff who had been minimally or not at all involved in Quest 
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events. Finally, Quest team members completed the Reflective Assessment questionnaire (see 
Appendix D). 

Pre- and post-test scores on the School Professional Staff as Learning Community (see 
Appendix E) were analyzed to discern if case study schools had become more like professional 
learning communities over the course of their participation in Quest. This instrument was first 
administered to all network schools in December 1 997, and again in November 1 999 as the project 
drew to a close. The surveys were sent to a contact person at each network school, who distributed 
the instruments to faculty, then collected and returned completed surveys to Quest staff. 

Another instrument completed by Quest participants at the close of the project was an 
Innovation Configuration Checklist detailing the essential components of Quest as well as variations 
thereof (see Appendix F). All network participants in attendance at the February 2000 rallies were 
administered the Checklist, including those from the four case study sites. 

Other data sources included achievement data from state-mandated standardized tests as 
available and data gathered during school participation in Quest sponsored activities, including 
technical assistance visits and project events. 

Identical instruments and individual and group interview protocols were used across the four 
case study sites to allow for comparative analyses. 

For this document, Quest staff met on October 18, 2000, to discuss what lessons they 
believed they had learned over the course of the project. The evaluator grouped these comments by 
theme and then conducted meta-analyses of formative and summative data, staff communications, 
and other project documents for corroborating or disconfirming evidence. 

The primary audience for this report is Quest staff; it is intended to provide them a final 
evaluative account of the project, with an emphasis on making clear the implications of their work 
for future endeavors. Other audiences include representatives of AEL’s funding source, the U.S. 
Department of Education’s Office of Educational Research and Improvement (OERI), and 
policymakers, school administrators, teachers, education researchers, and others interested in 
strategies to support continuous school improvement. 

The purpose of this document is to summarize the most important lessons learned by staff 
from their experience of developing and facilitating the school improvement network. It is intended 
to provide staff and others concerned with education a description of the challenges and successes 
of one Research and Development-based effort to support schools undertaking reform. 
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LESSONS LEARNED 
Leadership 

Quest staff reported that their impressions of the importance of school leaders to the success 
of reform implementation had been confirmed through their experience with Quest. Schools with 
supportive leadership found ways to attend various network events and enact school improvement 
projects. Those with less supportive leaders struggled to continue their involvement in Quest or 
terminated their involvement in the project. 

A study conducted during the project on the factors school staff thought had been most 
important to their school’s initial and sustained involvement with the network revealed that 
participants found administrative leadership to be the most significant catalyst (Howley-Rowe, 
1999a, p. 9). Illustrative comments from study participants include: 

"I feel like right now [our involvement with Quest] is really high because our principal is 
really actively involved, so therefore if a principal is actively involved and you know you 
have your school that’s going to follow." 

"I think the people at the administrative level are absolutely critical, because a student or a 
parent or even a teacher probably can’t make it go once it gets back home. And so I know 
we need our administrative person, and we’ve got one with a lot of energy now, and I think 
it’s going to make a huge difference." 

"I think the teacher can keep it alive but maybe couldn’t have made it really push. But 
having a committed administrator there is just really important." 

"If you don’t have the administrator, you have nothing at all." 

Other comments suggesting the significance of building-level administrative support for 
sustained involvement in the Quest network also referred to such administrators’ willingness to 
secure or broker funding, released time, and substitute coverage to enable participant attendance at 
project meetings (Howley-Rowe, 1999a, p. 10). For instance, 

"I think what makes it easy for us to be involved is that our principal is totally in agreement 
and dedicated to the situation, and so that’s never a question as far as freeing time to attend 
meetings or expenses or whatever it takes. She thinks it’s a good thing, and we are 
committed to do this." 

"I think it really has a lot to do with our staff and the fact that they can find money. Our 
principal can find money in places that we have no clue. We do not ask. We’re just glad the 
money’s there and she pays for us to come and it really helps. Because if we had to pay for 
it out of our pockets, I really don’t think as many people would be able to come." 
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"Again, I think resourcefulness [is important to involvement], I think, you know, having the 
principal back this really makes things possible. Because as I see it, and I’m on the outside, 
it costs the school money to be involved and all the teachers who are here . . . had to find 
money to find substitutes that are there with our children today and tomorrow. And just 
being here ... so, to stay at the hotel and the food and everything . . 

"While we can’t get [our principal] to come [to Quest events], he’s willing for us to go, and 
he’s very supportive, and he helps us try to find funding and substitutes and all those kind 
of things." 

Teachers at Bowman Elementary, in addition, suggested that their participation in Quest was 
encouraged because the principal had a history of commitment to reform and innovation: 

Teachers reported that the school’s mission and attendant work ethic were promoted by 
Principal Fred Vickers'. "He shares. And he doesn't make us feel like we are wrong or we 
are beneath him or anything. He makes us feel that we are really important and we are 
because if you look at your teachers, teachers can either make or break a principal," reported 
a teacher. Likewise, other teachers corroborated that Principal Vickers encouraged systematic 
analysis of teacher innovations as a vehicle for enhancing teaching and learning: "Mr. 
Vickers will allow us to use anything that we can. He won't say, "Rhonda, that's wrong.’ or 
‘Rhonda, I don't think you should use that.’ He'd say, ‘If you could tell me why you would 
like to use this . . . how you’re going to help your students.’ And sometimes he lets us fan 
out on our own and maybe this wasn't the best book to use. And I . . . again it goes back to 
that top person. That top person has to have a high vision of what that school is going to be 
all about." 

Yet innovation is expected to take place within the strictures of the school goals. As 
one teacher reported, "Mr. Vickers is very goal directed. So, there are, you know, goals are 
pretty well spelled out. Very focused and then he... I gave you my example I think yesterday 
about trying something in kindergarten and him saying, ‘Okay, but make sure you know 
where you start and where you stop.’ Which kind of goes along with the attitude of being . 
. . you know, having goals and then collecting data to see if you meet some of those goals. 
So, I think that . . . that's a strength of him as an administrator in setting kind of the target 
for our school" (Howley-Rowe, 2000a, p.9). 

Networks 

According to Quest staff, the use of the network to support continuous school improvement 
has been beneficial. One benefit has been the way in which practitioners were able to learn from each 



'Individual and school names in this report are pseudonyms to protect the confidentiality 
and anonymity of participants. 
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other rather than from researchers, state department of education staff, or other entities external to 
the life of a school. Characterized by some researchers as professional learning community (e.g., 
Hord, 1 997; Ryan, 1 995), Quest participants modeled strategies for each other, shared resources, and 
discussed issues of educational significance. They met at regular intervals throughout the year, at 
fall and spring rallies and summer symposia. Formative evaluation data suggest that participants 
forged meaningful and fruitful professional friendships over the course of their participation in 
Quest. 



Learning occurred between individuals and on a schoolwide basis, as when Saint Margaret 
staff presented their technology strategies to Bending Knee staff (Howley-Rowe, 2000b). The 
principal noted that teachers had begun to integrate technology into instruction, despite her earlier 
reluctance to make technology a priority. She elaborated, “I knew we were weak in technology . . 
. and frankly I didn't care. I didn't really believe that technology could do a whole lot to change 
instruction. But ... in the fall, when my staff went to the rally and Saint Margaret did the workshop, 
they were calling me — you know I didn't attend — and they were calling me and saying, ‘If we could 
be doing this in our classroom . . .’ So we had them [Saint Margaret staff] in here in February, and 
they did a training for the staff . . . I've seen a tremendous difference in lesson plans and utilization 
of technology in the classroom. And I've even come to believe that technology is a good tool.” 

Similarly, two Bending Knee staff members noted that new ideas and strategies were far 
more convincing when learned from other schools with experience implementing them. According 
to one such interviewee, “One or two people can read [about an innovation] and say, ‘Oh yes, let's 
do this. This is great. Let's try this,’ and try to convince the rest of the staff, for example. But it's 
never as impacting and as convincing as if you are networking with somebody who is already doing 
this process and it's worked.” Colorfully, one participant said, “It helps those who are hanging on 
the pier screaming and shouting kind of take a step over into the water.” 

Another strength of the network, according to project staff and Scholars program participants, 
was the diversity of its membership. As the following excerpt from the evaluation of the 1998 
Scholars colloquium reveals, participants found the multiple perspectives shared through the network 
to be useful to their professional growth. 

Not only did the Scholars appear to value the inclusiveness of Quest, they also thought that 
such diversity enabled them to analyze issues from a variety of perspectives. For instance, 
during one discussion, an administrator suggested that Quest staff and Scholars “need to sell 
the value of networking of local schools.” Then, the administrator of a small rural school 
described the difficulty of such an endeavor, noting her perspective that rural schools tended 
to be more competitive with one another. This exchange provided Scholars the opportunity 
to think about how facilitating networking might differ across rural and urban or suburban 
districts. Similarly, one Scholar observed that high school teachers seemed generally less 
collaborative than elementary teachers. The Scholars then had a brief exchange about why 
this might be. Again, the diversity of participants facilitated comparisons of multiple 
viewpoints. In terms of Quest itself, one Scholar said, “We all bring experience to Quest. 
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We’re open to learning. That helps us attain new ideas.” Another described the project as 
allowing people of diverse standpoints and experience to “be on the same playing field.” 
Thus, “see[ing] through others’ eyes” seemed to be an important part of the Quest 
experience. 

“Not only did the Scholars think that diversity was central to their Quest experiences, 
they valued diversity. For example, one participant said during a discussion that 
collaboration was facilitated by “holdfing] the idea that there are other ways of thinking and 
doing.” Her statement suggests a belief that diversity enhances productive work. One of 
Quest’s strengths, reported a Scholar, was that it offered a “different perspective.” Again, 
diversity is framed as a valuable attribute, providing network members analytical remove 
from their ongoing work (Howley-Rowe, 1998f)." 

At the same time, the experience of diversity also allowed participants to find common 
ground, focusing their attention on seeking solutions to their similar issues. As one Bowman 
Elementary interviewee said, "I'm sure every school has . . . some population maybe or some . . . 
group of students that we feel like are not reaching their fullest potential. And it's really easy for us 
as teachers to say, ‘It's because, you know, this . . . They're low income or whatever.’ I think that 
Quest has kind of challenged that ... I mean, every school has their own population that may be 
harder to reach and they have all done it. You know, sharing the success stories from other schools 
makes you say, well, they have done it with these obstacles in their way. You know, there's no 
reason why we can't do it too" (Howley-Rowe, 2000a). 

Bending Knee interviewees also made comments suggesting that the network allowed them 
to share experiences and ideas with staff from schools very different from their own that were 
nonetheless facing similar issues (Howley-Rowe, 2000b). Said one Bending Knee teacher, “I think 
the liaison . . . with schools who are probably even bigger than us, maybe even urban . . . who have 
the same difficulties and the same problems we have and discovering what solutions they use to 
overcome those problems helped us think [how to] overcome what problems we were going 
through.” 

Similarly, the inclusion of students in the Quest network provided important opportunities 
for administrators and teachers to hear their perspectives. As one Xavier teacher explained, “I think 
it has improved my relationship with other teachers and students here in school. I’ve got to know a 
lot of students that have gone onto the Quest team. This is such a large [school], 3000 students here, 
and some of these students I never knew until we went to Quest . . . Now I am involved in some of 
their programs. They come to me saying, ‘Would you like to do this? Would you like to do that?’ 
It has helped me branch out into the school” (Howley-Rowe, 2000c). DIAD was one Quest process 
participants believed successfully included student voice in improvement efforts. Said one Bowman 
Elementary teacher, "I really like the Data in a Day last year. It involved so many people. It involved 
the children here at school. It involved teachers here at school. It involved community people here. 
And there was such a sharing and such a cooperation that that could [not] help but benefit the total 
atmosphere of the school." 
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Quest staff indicated that their own participation in the School Change Collaborative (SCC), 
a national partnership coordinated by the Northwest Regional Educational Laboratory (NWREL), 
provided them with knowledge, skills, resources, and professional relationships they would not 
otherwise have accessed. Because Quest and SCC work focused on similar goals and philosophy, 
project staff themselves felt part of a larger school improvement network. As one staff member put 
it, "We were part of a network." 

The project director described in a personal communication the benefits of belonging to the 

SCC: 



A lesson learned for AEL's Quest project is the power of working with other labs/researchers 
and practitioners. The strategies employed and developed by the SCC have had immense 
impact on what we chose to do in the Quest project and earlier in LUSIE. I think that we 
have learned that you get better when you work with others who can help you move your 
own blinders a little farther apart. This applies to staff as well as the principals who are our 
partners. Having our own learning community at the SCC level has been an important piece 
for our own learning . . . DIAD was actually developed during the preceding five-year 
contract period, but we as staff did not really adopt it as a technique we were willing to use 
with our clients until our own staff were able to buy into and understand DIAD. ..to see its 
usefulness, to become advocates for three of the four techniques. We adopted those 
processes in conjunction with our partner principals who carried those techniques into their 
schools for the same reasons that staff carried the techniques into the rallies ... I think we 
learned that this was a successful collaboration because of the goodness of fit between 
NWREL and AEL's work in this particular project" (September 28, 2000). 

Multiple Strategies to Support School Improvement 

Quest staff reported the value of offering participating schools a variety of structures and 
forums to support school improvement. Site visits, for example, were said to have been valuable both 
for project staff and network members. Staff reported that such visits provided them with richer 
information about the schools and connected them more fully to participants. Technical assistance, 
facilitation of improvement processes, data collection, and semi-structured discussions with school 
staff were activities that enhanced relationships and gave Quest staff clearer understandings of the 
issues confronting schools. 

Participants, too, found the various support strategies helpful. The ways in which Quest 
supported school improvement, such as rallies and site visits, was among the five factors rated most 
important to schools’ continued involvement in the network (Howley-Rowe, 1999a). Several 
participants in the 1998 Scholars colloquium reiterated the importance of having ready, accessible, 
and meaningful support from Quest staff. One Scholar noted that the site visits, or co-ventures, had 
been helpful because “they pushed us to move more quickly” on various projects. Scholars praised 
the co-ventures, or site visits, in which Quest staff visited schools, collected data, planned or 
conducted a technical assistance activity, and later returned feedback to the school. This feedback. 
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one participant submitted, was useful because QUEST staff s “mission is only school improvement, 
not accreditation or testing . . . there are no hidden agendas.” Part of the value of this network 
support activity for this Scholar was that it provided some assessment that was unconnected to any 
potentially punitive or political relationship (Howley-Rowe, 1 998f). 

Participants often described rallies as energizing, informative experiences (e.g., Howley- 
Rowe, 1999b). Summer symposia were likewise received favorably by attendees (Howley-Rowe, 
1998g; Parrish & Howley-Rowe, 2000). 

Quest staff reported that their experience with Quest affirmed their perception that one 
reform strategy cannot meet the needs of all schools. During the project, staff offered participants 
myriad improvement perspectives and techniques from which to choose. Although the network 
allowed quite different schools to find common educational ground, the improvement projects they 
undertook as a result of their participation in Quest reflected their particular and local concerns. In 
addition, site visits gave schools the opportunity to receive individualized technical assistance from 
project staff. 

Another lesson learned by project staff was that intensive effort and substantial staff time are 
required to support schools as they undertake reform. Quest staff devoted much time to the 
development, coordination, and facilitation of site visits and individualized technical assistance, in 
addition to wider network efforts and project documentation. Some schools would have preferred 
more, and more intensive, technical assistance than was provided, but project staff were not available 
to do so, although the Quest network was not large. The project listserv, originally developed to 
support communication and exchange of resources among members, did not appear to approximate, 
let alone replace, the individualized assistance offered by Quest staff during site visits. In sum, Quest 
staff learned that school improvement necessitates significant attention to local school concerns and 
priorities, which in turn requires adequate project staffing. 

Parent and Community Involvement 

In addition to faculty and administrators, parents and students were invited to participate in 
the Quest network in an effort to broaden the community of participants and model the inclusion of 
diverse members of school communities. Parents reported appreciating their participation. One 
parent noted in a brief interview during a February 1998 rally that whereas she had felt out of place 
and unsure of her role in the previous rally she attended, she now felt much more at ease (Howley- 
Rowe, 1998d,p. 14). A parent contributed during the 1999 Scholars colloquium, "[Thereis] nowhere 
else parents would be privileged to sit down and talk about these things" (Howley-Rowe, 1 999e, 
P-10). 



However, collaboration across role groups did not always proceed smoothly. During the last 
day of the 1998 colloquium, Scholars were asked to collaborate with each other in the revision of 
creeds Quest participants had developed at an earlier rally (Howley-Rowe, 1998f). In one of the 
groups, a parent objected to the use of the phrase “instructional delivery,” adding that she would 
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prefer a term that had more “student-centered” connotations. Two of the educators participating in 
this group quickly dismissed her concern with the argument that the phrase was “widely understood 
within education,” an example of educational jargon. The group did not fully explore the parent’s 
point, nor did they pause to consider the implications such wording may have for an endeavor that 
aims to be collaborative. Instead, the two educators may have reinforced, at least in the parent’s 
mind, the notion that education belongs to educators, that educators are the experts in this arena to 
the exclusion of others. 

Parents themselves sometimes expressed their perception that they were peripheral to the 
work of Quest. One parent attending the November 1998 rally (Howley-Rowe, 1999b, p. 23) said, 
“I think this is a really big workshop for the teachers . . . [there are] good ideas ... but it’s for the 
teachers.” This parent, then, felt the content of the rally was more appropriate for teachers and less 
so for others. Another parent reported, “I’ve been on the Quest team for a year and haven’t yet found 
my place on it. We all do things to support the school, but it’s outside and separate from [the 
school].” 

Quest schools made significant efforts to include students and parents in project activities 
and meetings. They had less success with inclusion of community members. Schools struggled to 
reach out to community members and to define the roles community members might play in 
continuous school improvement. As a result, few such individuals became involved in the project, 
and none for an extended period of time. 

Logistical and Financial Support 

Quest staff noted that although the network experience was enriched by including 
participants from all four states in AEL’s region, finding centrally located meeting places was 
sometimes a challenge. In addition, such regional meetings entailed increased travel expense for 
some network members. 

For this reason, and because many schools struggled to locate funds to support their 
participation, Quest staff offered financial assistance to those signing a memorandum of 
understanding concerning their ongoing involvement. Project staff indicated that such financial 
support proved quite important to some schools, who used the stipend to pay for travel to Quest 
events and to hire substitute teachers to cover classes for network members attending rallies. For 
instance, one high school focus group interviewee noted the importance of funding received from 
AEL to support continued participation in the project: "The other factor is the money provided by 
AEL. If we didn’t have the money, I wouldn’t be here" (Howley-Rowe, 1999a, p.10). The study of 
factors supporting or hindering schools’ ongoing participation in Quest reiterated that lack of 
financial resources was often an impediment. 
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Evaluation 

Proj ect staff intending to impact and/or evaluate student achievement should become familiar 
with the various tests mandated by the states or districts involved. There tends to be little or no 
comparability between standardized tests, which limits the sorts of analyses possible. Case studies 
of four Quest network schools were conducted as summative evaluation of the project in part 
because standardized test measures were not comparable across the participating states. 

Moreover, project staff should become aware of how achievement data are made available 
to individual schools: on disk; on paper; in summary format by school, grade, or teacher; or as 
individual student scores. If the project evaluator wishes to obtain student achievement data, he or 
she should begin negotiations with participating schools early in the project, because schools 
sometimes do not store their data from previous years. A protocol to formalize, or at least structure, 
the exchange of data might be helpful. Even a project as sensitive to the needs and concerns of 
schools as Quest, and as nonprescriptive in its support to participating schools, should consider 
beginning the collection of student achievement data early in the life of the initiative. Although 
schools may focus upon different priorities, they ultimately aim to help their students learn; such 
learning should be assessed to the extent possible. 

Nonetheless, project staff and participating schools should recognize that school 
improvement initiatives may impact student achievement to varying degrees. Fullan (1991), for 
example, describes an implementation dip, in which student performance drops during the 
implementation of change as faculty and students adjust to new routines, strategies, or assessment 
techniques. In addition, schools in the network tended to participate in other initiatives, further 
clouding evaluation. Although some gains in student achievement were found at the four Quest case 
study schools, the patterns were inconsistent and ambiguous. 

Quest staff who had also participated in the development of QUILT (Questioning and 
Understanding to Improve Learning and Thinking) became convinced of the necessity of dedicated 
staff for program documentation and evaluation. Whereas little process and formative evaluation 
were conducted with QUILT because an evaluator was not committed to the project, Quest has 
ample documentation of its development because an evaluator was assigned between .30 and .70 
FTE at various points throughout project implementation. 

Issues for Further Consideration 

Quest staff identified several issues for which lessons learned were ambiguous or could not 
be clearly summarized. For instance, the nonprescriptive nature of the project proved compelling to 
some participants and schools but less so to others. Xavier Senior High School learned of several 
processes through Quest, selected DIAL) as a means to analyze their block schedule, and then 
modified the technique to meet their own objectives. Bowman Elementary chose to use the Protocol 
process in an effort to improve their fourth grade writing scores, which had declined. Staff at Tinder 
Elementary accessed the Region IV Comprehensive Center at AEL through discussions with Quest 
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staff about troublesome math achievement. For these and other schools, Quest offered a variety of 
resources and tools that could be applied to meet school-specific needs. 

On the other hand, some participants reported that the project was equivocal or did not 
prescribe practice. Following the February 1998 rally for elementary schools, for instance, one 
participant suggested that "a more defined road map [of school improvement] would be useful" 
(Howley-Rowe, 1998e, p. 19). Twelve participants in the study of schools’ initial and ongoing 
involvement in the network reported that the open-ended quality of the project had served as an 
impediment to their participation (Howley-Rowe, 1999a). 

The Quest Framework for Continuous Improvement offered participants a theoretical model 
around which to plan school improvement efforts, which may have helped mitigate the confusion 
some participants felt. For example, participants were asked following the November 1998 rally to 
evaluate how well the review of the Quest framework “worked” for them (Howley-Rowe, 1999b). 
Twenty-six of the 35 (75%) who responded felt that the review had been, for instance, “vital” and 
“needed.” Other comments included “I am new to the program so it was helpful,” “I’m beginning 
to get the idea . . .,” “This was good not only for me because I had heard it before, but because there 
were new people,” and “Now I understand more what Quest is about.” And at the 1999 Scholars 
symposium, participants reported that reviews of the framework were useful (Howley-Rowe, 1 999e). 
"Revisiting the [framework] components helped me refocus," reported one Scholar. Discussing the 
framework components "means so much more over time" to another. 

Another issue about which a clear lesson could not be discovered concerned the voluntary 
nature of participation in Quest. In other words, schools and districts were not required by larger 
governing agencies to be involved in the network. It was unclear to Quest staff whether this had been 
a liability or an asset for participants and the project. On one hand, those who became involved did 
so because Quest aligned with their school goals or philosophies in some way; their commitment to 
the project was their own rather than as a result of mandate. On the other hand, schools were not 
accountable to governing agencies to maintain their participation, rendering a few schools’ 
involvement perfunctory. Should Quest continue in some form, staff may want to consider 
examining the effect of mandatory or voluntary participation on involvement and attrition. 
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CONCLUSIONS 

Based on the observations above, evaluation data, and the experience of documenting Quest, 
the evaluator offers the following conclusions. 

Building-level school leadership proved to be a significant factor in the degree to which 
schools participated in the network. With leaders who were at least somewhat supportive of the 
Quest endeavor, schools were able to locate funding for travel and substitutes and to implement 
improvement efforts. Those without supportive leadership struggled to maintain their membership 
or left the project entirely. Although participants reported the network to have been valuable and a 
source of encouragement and information, the network alone did not seem capable of guiding those 
schools with leadership unsympathetic to Quest to the project’s school improvement perspectives 
or strategies. 

On one hand, schools often benefit from the local control they are able to exercise over 
instructional, curricular, governance, and finance issues. Moreover, school communities and staff 
are perhaps more likely to feel ownership of decisions in which they had a part in making. On the 
other hand, a school-level leader uninterested in school improvement may seriously deter staff 
efforts to implement reform. This appeared to be the case in several Quest schools, whose 
involvement was hampered. 

For those participants able to remain involved in Quest over the course of its existence, the 
network of schools inspired school improvement efforts through the combination of dialogue with 
other network members, research summaries, planing activities, action research initiatives, strategies 
for school assessment, and site visits. In addition, participants often learned from one another rather 
than from project facilitators; new ideas and strategies were viewed as more convincing when 
learned from network schools with experience implementing them. The schools most engaged with 
Quest provided technical assistance to sister schools during site visits, shared resources and 
information, and kept in contact via e-mail or telephone. 

Another strength of the network included its diverse membership. Schools varied in their 
size, rural or urban designation, state, and demographic composition. Participants themselves were 
diverse, including administrators, teachers, other school staff, parents, students, local school 
improvement council members, and sometimes community members. Network members themselves 
were not especially racially or ethnically diverse, which is more likely a reflection of the 
demographic make-up of AEL’s region than of insufficient Quest attempts to enhance the diversity 
of the network. In any case, participants reported that the mix of perspectives and experiences in the 
network enhanced their sense of common educational ground. 

Quest staff, too, benefitted from participation in a broader network of researchers and 
practitioners, the SCC. The collaborative offered project staff resources, relationships, and strategies 
to support their own efforts to facilitate school improvement via Quest. Because the SCC and Quest 
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held common aims and perspectives, project staff were able to situate their work within a larger 
context by sharing their efforts with colleagues from across the country. 

The use of multiple strategies to support schools undertaking improvement initiatives appears 
to have been useful. For some schools, site visits were quite valuable, providing them occasion to 
view strategies first-hand. Other schools found rallies energizing and resource-rich. Scholars 
reported that the opportunity to participate in research, writing, and presentation with AEL 
researchers was a rare and informative experience. Summer symposia gave schools a forum to which 
they could bring larger numbers of faculty, staff, and parents and sample a wide variety of techniques 
to support student learning. 

Most parent participants in Quest appreciated very much the opportunity to provide input into 
their schools’ reform work. Moreover, project rallies allowed staff and parents to discuss issues of 
broad educational import in an intellectually safe environment. Nonetheless, Quest staff were 
challenged to involve parents meaningfully; not only are parents’ roles in schools changing, but 
school staff themselves sometimes continue to struggle with maintaining their traditional turf. 

Financial support proved to be an important factor in enabling schools to continue their 
participation in the network. Although likely not the only factor nurturing involvement, the 
additional funding enhanced the ability of some schools to remain in Quest. 

Quest processes were thoroughly documented because a staff member was dedicated for that 
purpose. Not only was formative evaluation conducted, summative evaluation of four Quest sites 
was also completed. One lesson learned during this process is that the format of achievement data 
available for analysis varies widely across states and schools; it behooves evaluators to learn early 
in projects what is accessible. Another lesson is that student achievement is effected by many and 
diffuse factors other than individual reform efforts. For this reason, evaluation of student 
achievement data should be interpreted with caution. 

Data were not collected to explore the relationship between voluntary versus mandatory 
participation and degrees of involvement in Quest, nor was such a design feasible given the nature 
of AEL’s work. Nonetheless, project staff would have been interested to examine whether type of 
participation had differential effects on degree of participation. 

Quest staff also continue to have an interest in the relationship between the 
nonprescriptiveness of the project and levels of participation. For some schools, the way in which 
Quest supported a wide variety of improvement efforts to best address the needs of individual 
schools was useful. Quest provided such schools the freedom to explore myriad reform strategies 
and choose those which most closely aligned with their local goals and perspectives. Other schools, 
however, found the project approach ambiguous. These schools might have preferred a lock- step 
reform model, such as some supported through Comprehensive School Reform Demonstration 
program funds. 
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To conclude, Quest appears to have generated knowledge that will serve project staff well 
as they continue their work to improve schools in AEL’s region. For instance, staff will likely 
consider seriously strategies to gain the support of school administrators for any projects similar to 
Quest undertaken in the future. The use of networks may also be continued, given the success with 
which it supported the school improvement efforts of participating schools. And staff will continue 
to use evaluation as a means to document and refine the efficacy of future endeavors. 
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Quest for Quality Learning Communities 

A Program for Continuous School Improvement 



School improvement is challenging work; to be 
effective, it must be continuous. Improvement is 
not a single act or program; it is a process of al- 
ways wanting to learn more about how better to 
help all students achieve at higher levels. Improve- 
ment is visionary; it involves risk-taking, uncer- 
tainty, and a rejection of “doing what we’ve 
always done.” Most of all, improve- _ _ _ Energy 

ment requires more than indi- A 

vidual effort: it is a collabo- 
rative endeavor that en- 
gages and responds to 



dividual responsibility for better performance, col- 
legial sharing and support, and thoughtful reflec- 
tion on practice. 



the diverse 
within an 
community. 

Teams from 20 
schools in a four- 



Stemming from these values is a clearly defined 
vision of student excellence that is shared by all 
members of the school community. A strong 
learning culture encourages both stu- 
dents and teachers to choose con- 
tinuous improvement as a way 
of life in their school. Mem- 
bers of the school com- 
munity connect to one 
another through a 
shared commitment 
to improved learn- 
ing conditions for 
all. Shared leader- 




state region now 
collaborate with 
staff from the Ap- 
palachia Educa- 
tional Laboratory 
(AEL) to study and 
learn together in the 
Quest project, and each 
school takes a slightly different 
path. For example, one school tar- * * * 

gets increased parent involvement; an- 
other hopes to raise the level of student thinking 
through teachers’ working together and coaching 
one another; a high school improves teaching by 
listening to what students say about how they 
learn best; other schools focus on specific curricu- 
lum areas such as writing or science education. 

The Quest framework unifies their thinking about 
school improvement. These core values offer a 
blueprint for continuous progress: ongoing ques- 
tioning of practice, high expectations for all, in- 



ergy 



ship encourages and 
enables everyone to 
assume responsibility 
for making a positive 
impact on the school 
community. Shared goals 
for student learning moti- 
vate individuals to improve 
their performance and help focus 
the energies of the entire community. 

The collection, analysis, and use of student as- 
sessment data sustains continuous improvement, 
providing a measure of the effectiveness of the 
community’s efforts. SMART learners are Suc- 
cessful, Motivated, Autonomous, Responsible, 
and Thoughtful. Fully equipped to become life- 
long learners, they are ready for life and work in 
the 21st century. In short, continuous improve- 
ment spawns the energy and excitement neces- 
sary to transform a collection of individuals into 
a true learning community. 



Goals of the Quest Project 



1 . Connect with colleagues. By serving on a Quest leadership team, participants connect with others on their school 
team, forming bonds that enhance working relationships. In addition, Quest teams connect with teams from 
other schools, districts, and states, allowing everyone to learn from others' experiences. A listserv, inquiry@ael.org, 
facilitates connections across the network. 

2. Create a learning community. Teams become part of the Quest net- 
work learning community with the expectation of recreating this ex- 
perience in their own community. 

3. Connect vArix concepts and stories related to continuous school im- 
provement. At Quest rallies, the Quest framework is a source of study, 
dialogue, and sharing among teams. 

4. Create personal and shared meaning. The Quest network places a 
high value on processes such as reflection and dialogue, which lead to 
deeper understandings of continuous improvement. 

5. Commit to continue learning with this community Quest schools 
have made a three-year commitment to study and learn together, with 
a focus on improving student achievement. 

6. Commit to continue the Quest back home. The rubber hits the road at schools, not at Quest events. AEL helps 
school teams take their learnings home and apply them for the benefit of students. Site visits, called Co-Ventures 
in Learning, provide opportunities for AEL staff to visit each school, in order to better understand the context of 
that schools efforts, and tailor assistance to the school s needs. 



What Is a learning community? 
“Learning communities are essen- 
tially communities of Inquirers . . . 
sustained by a continued commit- 
ment to share this Journey of 
exploration with one another on 
matters people care deeply about” 
(Ryan, 1995). 



Peter Senge et al. (1 994) write that a 
learning organization “is a place 
where people continually expand 
their capacity to create the results 
they truly desire, . . . and where 
people are continually learning how 
to learn together.” 



The Quest project hopes to achieve results at three different levels: 

• For individuals, sharing leadership on a Quest team leads to more reflective practice and renewed under 
standing of the concepts that support continuous improvement. 

• For schools, Quest will provide motivation and support for ongoing and/or new school-based initiatives to 
improve teaching and learning. 

• For the Quest network of schools, our collaborative learning and research will yield stories, insights, 
processes, and products — all of which will be helpful to the broader educational community. 
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contact Sandra Orletsky or 
Beth Sattrs (sartesb@ael.org) at 



R O. Box 1348, Charleston, WV, 25325 
800-624-9120 

VMnvw.ael.org/reL/quest/index/htrnj/ 
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APPENDIX B: 



Completed Evaluation Standards Checklist 



Checklist for Applying the Standards 



To interpret the information provided on this form, the reader needs to refer to the full text of the standards as they appear in Joint 
Committee on Standards for Educational Evaluation, The Program Evaluation Standards (1994), Thousand Oaks, CA, Sage. 

The Standards were consulted and used as indicated in the table below (check as appropriate): 



Descriptor 


The Standard was 
addressed 


The Standard was 
partially addressed 


The Standard was 
not addressed 


The Standard was 
not applicable 


U1 


Stakeholder Identification 


X 








U2 


Evaluator Credibility 


X 








U3 


Information Scope and Selection 


X 








U4 


Values Identification 


X 








U5 


Report Clarity 


X 








U6 


Report Timeliness and Dissemination 


X 








U7 


Evaluation Impact 


X 








FI 


Practical Procedures 


X 








F2 


Political Viability 


X 








F3 


Cost Effectiveness 


X 








PI 


Service Orientation 


X 








P2 


Formal Agreements 


X 








P3 


Rights of Human Subjects 


X 








P4 


Human Interactions 


X 








P5 


Complete and Fair Assessment 


X 








P 6 


Disclosure of Findings 


X 








P7 


Conflict of Interest 


X 








P8 


Fiscal Responsibility 


X 








A1 


Program Documentation 


X 








A2 


Context Analysis 


X 








A3 


Described Purposes and Procedures 


X 








A4 


Defensible Information Sources 


X 








A5 


Valid Information 


X 








A6 


Reliable Information 


X 








A7 


Systematic Information 


X 








A8 


Analysis of Quantitative Information 


X 








A9 


Analysis of Qualitative Information 


X 








A10 


Justified Conclusions 


X 








All 


Impartial Reporting 


X 








A12 


Metaevaluation 


X 









The Program Evaluation Standards (1994, Sage) guided the development of this (check one): 

request for evaluation plan/design/proposal 

evaluation plan/design/proposal 

evaluation contract 

x evaluation report 

other: 

Name Caitlin Howley-Rowe Date 12/7/00 



(signature) 

Position or Title Research Associate 



Agency AjjHn — ^ nc 

Address 1031 Quarrier Street Charleston. WV 25301 



Relation to Document 



Author 

(e.g., author of document, evaluation team leader, external auditor, internal auditor) 
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